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F inding answers 


Evaluation keeps an organization on course and prevents it 
from flying blind. Evaluation is simply the process of finding 
honest and objective answers for questions like: How well are 
we doing what we are doing? Evaluation is used to judge or 
measure attitudes and ideas, programs, projects and goals. 
These judgments have to be based on objective evidence and 
they have to be made systematically. 


Good evaluation meets people on their own ground. The ways 
people think and the interest they take in any subject will vary 
with their education and background. Standards used in 
evaluation should be those that will put the next improving 
step within easy reach of the group. It would be discouraging 
to apply the standards of the London or Broadway stage to the 
productions of an amateur dramatic group at a local church. 


The best evaluation is co-operative. Sharing the work of 
measuring progress leads all members toward deeper un- 
derstanding and greater desire to improve than if some higher 
authority carries out the evaluation for them. 


Useful evaluation considers every angle. Evaluators should 
consider everything related to the activities or undertaking 
being evaluated. The key factor may turn out to be one of the 
least obvious and something no one had considered. 


Business-like evaluation is continuous. No community 
organization or committee should wait until failure looms 
before they apply systematic evaluation. Different techniques 
may be used at different stages, but evaluation should be 
always going on. 


Effective evaluation uses frank and probing questions. 
Evaluation must criticize and diagnose not just activities, but 
also the ultimate goals. Goals we set for ourselves are not 
sacred. They should be reviewed periodically in the light of 
new experience and information. The purpose of evaluation is 
more than just finding out whether a desired result is, or is 
not, being achieved. Its aim is also to find out why a given 
course of action does, or does not, produce the desired result. 


All evaluation leads to action. The only purpose of evaluation 
is to improve programs and activities aimed at goals, and to 
improve the goals themselves in the light of individual and 
social needs. 


Who evaluates? 


Because participation in community organizations is 
voluntary, it is good to think of evaluation as a group activity. 
Members should evaluate their own actions. No outsider 
should decide whether they have succeeded or failed. Outside 
experts are useful mainly in helping to devise methods for 
collecting the required kind of information and in helping the 
group to analyse and interpret that information. The final 
judgment or evaluation should be made by the group. 


What is evaluated? 


In a community organization, evaluation is often concerned 
with the organization itself, or the effects of the organization's 
work on its members and on the community. 


The organization needs to know how well it functions. What 
proportion of members attend meetings regularly? How well 
do members participate in activities? Is the organization 
making full and good use of all its resources of leadership and 
finances? How effective are its committees? 
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Evaluation of activities in terms of their effect on members will 
help keep the programs in tune with members’ needs. Are 
there changes in the knowledge of members or in their 
behavior as members, neighbors and citizens? Are they 
growing more mature and objective about their own and 
others’ behavior? Harmony, or lack of it, among the members, 
and the effectiveness of the methods being used to deal with 
tensions are other factors that may be tested. 


The impact of the organization on the community may also be 
measured. How far is the organization helping to meet 
community needs? Does it have an appreciable effect on the 
co-operation among groups? 


Evaluation and goals 


Before they can determine the effectiveness of meetings, 
programs or activities, members must have clear targets or 
goals. If these goals are too general, evaluation will be dif- 
ficult. For instance, a group that plans a series of meetings “to 
promote good citizenship” will find it hard to measure any 
progress. But a series of meetings planned “to introduce new 
Canadians to community” has a well-defined target. 


In setting goals there are several questions to bear in mind. 


LJ Are the goals in line with aims, objectives and policies of 
the organization? — 


[] Are they acceptable to the members concerned? Are they 
expressed in words members really understand? 


L} Are they clear enough to be practical? 


[] Is the idea contained in the goal in line with the way most 
members think? Does it set too high or too low a standard for 
this group? 

[] Are the goals attainable? Can they be measured? 


Evaluation techniques 


There are no hard and fast rules in evaluation. The 
measurement of progress is simple enough if all we have to do 
is collect statistics - for example: 


— number of people who buy tickets for a dance or enrol in a 
workshop. 


— number of people who respond to a fund-raising cam- 
paign, 
— totals of donations by days or weeks, 


— number of votes for and against questions submitted to 
members. 


But more often than not, community groups also want to find 
out things that cannot be measured so easily. For example: 


— whether members are satisfied with particular programs, 
policies or administrative procedures, 


— what members think about a community or group 
problem, 


— whether members are ready for a proposed change. 


Our most valuable source of evidence in evaluating com- 
munity group activities is the members’ freely expressed 
opinions. Techniques for getting this evidence can vary from 
informal remarks to elaborate surveys. What is most im- 
portant is that the problem should be approached from 
various angles so that evidence from one source may be 
compared with the evidence from another. This approach 
increases the objectivity and soundness of the conclusion. 


There are many devices for getting to know the views and 
feelings of a group. One of the simplest is the buzz-group, a 
technique for involving the members in program planning. At 
each meeting, the members may form groups of six or eight 
without leaving their places. Each buzz-group chooses a 
spokesperson and after about 10 minutes of discussion, he or 
she will report the program suggestions of the group to the 
general meeting. It is the spokesperson’s responsibility to 
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ensure that everyone in the small group has a chance to 
contribute. 


Many organizations use post-meeting reaction slips. At the end 
of the meeting or program to be evaluated, a copy is handed to 
each member so that he or she can indicate, without signing a 
name, reactions to specific questions. Great care must be 
taken in framing the questions so that they will not be an- 
swered thoughtlessly or automatically. 


The answers should also form a useful basis for planning 
future activities. Often some form of rating scale is needed so 
members can register shades of opinion between “yes” and 
“no”, or say how much they liked or disliked a program, plan 
or idea. Excellent-good-fair-poor is a rating scale of four 
parts. A scale to rate the reaction of a group may also be 
divided into five parts, or three or ten, for that matter. A 
reaction scale in five parts might be attached to a question 
such as “How satisfactory did you find this lecture?” 
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Evaluation in program planning 


Good programming is perhaps the most important single 
factor in the vitality of any community organization. Let’s 
consider an example. 


Suppose the chairman of a program committee is a former 
public health nurse with an enthusiastic interest in preventive 
health care. The nurse arranges a series of meetings — all 
lectures, on the topic in which he or she is interested. 
Qualified speakers are invited to lecture. The committee makes 
suitable physical arrangements for the meetings and gets out 
invitations and encloses pamphlets giving background in- 
formation. 


The speaker is highly trained and speaks at great length. 
During the first talk, the members are restless. At the second 
meeting, there is a drop in attendance. For the third meeting, 
only a handful appear. The series is an obvious failure. 


Applying some of the principles of evaluation, the failure may 
be analysed. 


(} Evaluation of the plan might have shown that the method 
of presentation was out of line with the members’ way of 
thinking. 

|] Through co-operative evaluation at the planning stage, 
the members would have had the opportunity to express their 
opinions. 

[] Continuous evaluation would have provided the warning 


signals, and suggested ways of saving the later meetings from 
failure. 


[J] Functional evaluation, at any stage, could have led to a 
change of the program content or presentation. 


Before deciding on their program plan, the committee might 
have asked members whether they would like a series of lec- 
tures on a single topic. If that had been acceptable, a list of 
topics could have been drawn up, with spaces for suggestions, 
and each member given an opportunity to rate the choices on 
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the list. The topics selected would then have been in keeping _ 
with the members’ wishes. 


After the first lecture, a general post-meeting questionnaire 
like the one shown might have been offered to the members. 


Assessment of program 
1. Please indicate your opinion of this program 
(a) poor fair good excellent 


(b) Comments 


2. Please give your suggestions for improvement 


Committee convenors and executive officers should apply the 
principles of evaluation to their own work. Their starting 
point is the question, “What should we be doing?” When an 
executive group has a clear, honest answer to that question, it 
may begin to collect evidence to show where they are in 
relation to the goal. 


The effectiveness of an executive can be measured in rated 
answers to key questions. 


LJ What proportion of the organization’s programs are 
initiated by members rather than officers? 


LJ How many members maintain their interest at a high level 
and participate actively in the programs? 


They may use some of the methods already suggested to find 
out the members’ opinions of their work. 


An example of evaluation 


Suppose a community group is planning to raise funds to 
purchase an encyclopedia for the school library. The 
evaluation process would work like this. 


1. Whats the target? 


The committee with the job of fund raising will probably first 
survey the need, and decide which encyclopedia to buy. If the 
group is convinced, on the basis of facts, workers available, 
and other factors involved, that it can raise the $1,000 needed 
to buy the fine encyclopedia the experts have recommended, 
they have set the goal. 


2. Where are we now? 


Once they have the goal, they have a standard against which 
to measure their progress. It will be important to know what 
results their methods are producing. A careful record of 
progress toward the objective will be useful in future planning. 


3. What activities will lead toward the goal? 


The committee must now decide what form their fund raising 
will take. They decide on a Christmas bazaar because they feel 
it will attract many people in the community. The committee 
begins the search for volunteers to provide the items for sale. 
After a week, the committee members report little success. An 
analysis of information also reveals that the Christmas bazaar 
will be poorly attended because other community affairs are 
planned for the same week. On the basis of this evidence, the 
committee now changes its course of action. It decides to plan 
for a community party after Christmas - a Valentine party in 
fact. The available facilities are surveyed, a program plan 
drawn up, publicity arranged and tickets distributed for sale. 


4. How are we doing? 


Here is where records of the effectiveness of various methods 
used in the past will be useful. Suppose, after 10 days the ticket 
sale is disappointing compared with the committee’s goal and 


expectations. The members will have to look for possible 
reasons. 


5. What does the collected evidence mean? 


A study of the record of the ticket sale, comparisons with past 
sales, and other evidence may show that few people are talking 
about the project or that some large group, such as teenagers 
are not interested. 


6. Are we doing what we should be doing? 


More canvassers, a telephone “blitz” and posters are arranged 
to give more publicity. The program of the party itself might 
be changed to include features to attract teenagers or other 
groups who have not been buying tickets. 


7. Can we reach the objective? 


If it becomes clear that none of the committee's efforts can 
possibly produce the needed $1,000, the goal for the project 
may have to be cut down, perhaps to half. The committee 
would then begin to plan another project to be carried out 
later. The objective for the new project would be the other 
$500 needed to buy the encyclopedia. The committee might 
decide to modify its objective in other ways. For instance, cut 
it by one-third and buy a less expensive encyclopedia. 


Another application of the steps 


For another example, suppose that at a Home and School 
Association meeting in November, some people are interested 
in setting up an adult education program. Evaluation will play 
an important role in the project. 


Before members decide their goals, they should try to find out 
the specific adult education needs in the community. They 
should consider the local facilities and whether leadership is 
available. They should decide how many classes will be held 


Eventually, they might offer three courses at classes held from 
7:30 to 9:30 on Wednesday evenings, beginning the second 
Wednesday in January and ending the third Wednesday in 
April (15 weeks) with a minimum enrolment of 15 people in 
each class. 


Having set their goal, the committee can measure their 
progress. Allowing for the Christmas rush, they have about six 
weeks to get the project underway. They might appoint a sub- 
committee to meet once a week to follow up registrations, 
arrange for staff and class-rooms and take care of routine 
details. 


Now, the committee may notice early in December that there 
is little enthusiasm for one of the courses. Perhaps only four or 
five persistent talkers had originally wanted it. Meanwhile, the 
committee has seen that another course will require three or 
four leaders to take care of the numbers registering for it. 


The committee must now find solutions to two problems. It 
can make a closer canvass for participants in the unpopular 
course, substitute a more appealing subject, or limit the 
project to the two popular courses. The association may go 
outside the community to find enough leaders for the over- 
subscribed course or they may adopt a study-group approach 
that requires no lecturers. 
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The committee may face decisions of this nature up to the 
week before the courses begin. As new evidence turns up, the 
committee will interpret it and modify decisions and actions in 
the light of new facts. Some association members may feel they 
are settling for less than the ideal goals originally set. 
However, only frustration will result from continuing to strive 
for objectives that are unobtainable on the basis of good, 
sound evidence. 


Once the classes start, a new evaluation cycle begins. Each 
instructor should apply the seven steps in evaluation to find 
out how well the classes are working toward the goal. The 
evidence they analyze and interpret is made up of the opinions 
expressed by members of the classes. When the classes are 
completed in April, the committee will do an evaluation of the 
entire course, methods of presentation, times chosen, facilities 
and suggestions for next year. In short, good evaluation is a 
continuous process. 
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Use of an observer 
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Committees and executive officers might also ask an ex- 
perienced outside observer to report after watching them in 
action for some time. A person who is not involved in the 
processes of the organization can often see more clearly than 
the participants. 


Questions the observer would ask are: 
— Does the group understand its purpose? 
— Isthe group progressing toward its objectives? If not, why? 


— Is the group making the best possible use of all the 
resources and facilities available to it? They may “belong” to 
the community and others, but can be shared. 


— Does the leadership of the organization help all members 
to participate and to take increasing responsibility? 


The observer’s report should be presented frankly and =ob- 
jectively. It is not necessary to refer to people by name or to 
criticize individuals. The observer’s responsibility is to describe 
the group process as he or she saw it. The group itself can draw 
the conclusions about the weakness or strength of its own 
procedures. 
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